iiiiiiiiiiii iiiiiiii iiin 

US006330602B1 

(12) United States Patent (h» Patent No.: us 6,330,602 bi 

Law et al. (45) Date of Patent: *Dec. 11, 2001 



(54) SCALEABLE WEB SERVER AND METHOD 
OF EFFICIENTLY MANAGING MULTIPLE 
SERVERS 

(75) Inventors: Eddie Ka Lun Law, North York; 

Biswajit Nandy, Ottawa; Alan S J. 
Chapman, Kanata, all of (CA) 

(73) Assignee: Nortel Networks Limited, Montreal 
(CA) 

( * ) Notice: This patent issued on a continued pros- 
ecution application filed under 37 CFR 
1.53(d), and is subject to the twenty year 
patent term provisions of 35 U.S.C. 
154(a)(2). 

Subject to any disclaimer, the term of this 
patent is extended or adjusted under 35 
U.S.C. 154(b) by 0 days. 

(21) Appl. No.: 08/965,929 

(22) Filed: Nov. 7, 1997 

(30) Foreign Application Priority Data 

Apr. 14, 1997 (CA) 2202572 

(51) Int. CI. 7 G06F 15/16 

(52) U.S. CI 709/224; 709/217; 709/226; 

709/249; 709/203; 370/428; 370/401 

(58) Field of Search 709/217, 218, 

709/219, 226, 203, 228, 227, 229, 249, 
252, 224; 370/428, 429, 401^09 

(56) References Cited 

U.S. PATENT DOCUMENTS 
5,748,897 * 5/1998 Katiyar 395/200.49 



5,761,507 
5,774,660 
5,774,668 
5,848,233 
5,850,517 
5,903,559 
5,933,606 
5,951,694 
5,963,944 
6,006,264 
6,047,323 
6,052,718 
6,070,191 
6,078,943 
6,092,178 
6,094,680 
6,097,882 
6,108,703 



6/1998 
6/1998 
6/1998 
12/1998 
12/1998 
5/1999 
8/1999 
9/1999 
10/1999 
12/1999 
4/2000 
4/2000 
5/2000 
6/2000 
7/2000 
7/2000 
8/2000 
8/2000 



Govett 395/684 

Brendel et al 395/200.31 

Choquier et al 395/200.53 

Radia et al 395/187.01 

Verkler et al 395/200.32 

Acharyaetal 370/355 

Mayhew . 

Choquier et al 714/15 

Adams . 

Colby et al 709/226 

Krause 709/227 

Gifford 709/219 

Narendran et al 709/226 

Yu . 

Jindal et al. . 
Hokanson . 
Mogul . 

Leighton et al 709/226 



* cited by examiner 



Primary Examiner — Robert B. Harrell 
Assistant Examiner — Stephan Willett 



(57) 



ABSTRACT 



A client-server architecture includes a plurality of clients 
and a plurality of servers. Information resources are repli- 
cated among the servers. According to one aspect, the 
invention includes an intermediary device called a "depot" 
sitting transparently between a client and a pool of servers 
which have the replicated information resources. The depot 
dynamically distributes multiple sessions contained in a 
client request among the servers. This architecture realizes a 
good granular scaleability of servers, and improved server 
throughput with a good response time. Multiple depots also 
realize robustness. 

20 Claims, 4 Drawing Sheets 
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SCALEABLE WEB SERVER AND METHOD lecture etc.). Alternatively, multiple servers can be used to 

OF EFFICIENTLY MANAGING MULTIPLE handle high rate of server transactions. 

SERVERS The multiple server approach has two immediate advan- 
tages: if a server fails, the session can be handled from other 

FIELD OF THE INVENTION 5 servers; also the total cost of multiple servers can be less 

Tne invention relates generally to distributed server sys- * an ' he °T °l° QQ whh 1 the t ec 3 uivalent Performance, 

terns which are used widely in the field of telecommunica- U 15 ther f fore ^ reseen that multl P le server s y stems wlll L be 

lions for information sharing. In particular, it is directed to in S reat demand t0 ^commodate an ever-increasing number 

a distributed server system which is scaleable and realizes ot user transacUons * 

high server performances. The distributed server system of Different architectures for multiple server systems are 

the invention includes an intermediary node between a client currently in use and are described briefly here. The use of a 

and a pool of servers. Domain Name System (DNS) server to distribute traffic 

among multiple servers was investigated at NCSA at the 

BACKGROUND OF THE INVENTION University of Illinois. FIG. 1 shows the DNS system. When 

15 a client 10 wishes to communicate with a server 12, at first 

The last few years have observed a phenomenal growth in it contacts the DNS 14, from which it obtains the IP address 

web (short for World-Wide Web or WWW, or Internet) of the desired scrver ^ client then ^ this Ip address to 

usage. This growth has demonstrated the value of wide-area communicate with the server. All clients perform the same 

information-sharing but, at the same time, caused a signifi- process unless they alrcady havc thc IP addreS ses of servers 

cant research interest m improving the performance of web 20 wim which they wam tQ connect vvhen there are a plurality 

systems. Recent studies show that the web consumes more 0 f servers which hold identical information, the DNS rotates 

Internet bandwidth than any other application. in a round robin manner through a pool of these identical 

At a macro level, a web system consists of three compo- servers which are alternatively mapped to the alias of the 

nents: (i) client, (ii) communication protocol, and (iii) hostname of one server. This approach has provided some 

server. Efforts are being made at each component to enhance 25 success in distributing the server load, however, it could not 

the performance of the overall web systems. balance the load among servers. Another problem with this 

At the client end, supports are provided to improve approach is that, once the IP address resolution is cached in 

response time by the following features: memory cache, disk the local memory, the client may never contact DNS. 

cache, allowing multiple simultaneous sessions and intro- Another system uses the HTTP level redirection capabil- 

ducing a proxy server for another level of caching. ity to move a transaction among multiple servers. FIG. 1 also 

The communication protocol between the client and the shows this mechanism. When a server 12 finds that it is 

server is HTTP (Hypertext Transfer Protocol) which always impossible to handle any extra traffic, it can redirect a 

assumes the existence of a reliable path layer underneath the transaction to another preselected server 18 and hence 

client and server. TCP/IP (Transmission Control Protocol/ 3S distribute the load. HTTP redirection is a common technique 

Internet Protocol) provides reliable data transmission using used for WWW load distribution. The implementation 

window flow control techniques. HTTP therefore runs on maybe simple and straightforward, but the redirection 

top of the TCP/IP layer. Asynchronous Transfer Mode requires a round trip delay between the client and server 

(ATM) is another transmission technique to handle broad- before the transaction is redirected to a different server, 

band multimedia traffic. It continues to grow steadily in the 40 Moreover, if the first server is already very busy, the 

communication world. High-speed ATM switches are avail- response delay will be even greater, 

able in the commercial market. The co-existence of the FIG. 2 shows another known system which switches the 

Internet and large-scale ATM networks is expected in the load based on the client IP address. Each client 20 goes to 

near future. ATM can provide wide-area virtual circuits, thus an intermediate device 22 which examines the originating IP 

facilitating geographical distribution of web servers. 45 address and decides where to forward the traffic among 

The HTTP has also undergone changes for performance multiple servers 24. IP address hashing is one of the possible 

improvement. It has been reported that multiple TCP ses- mechanisms to determine the server to which the traffic will 

sions per HTTP transaction is a major cause of performance be directed. This technique, however, lacks the dynamic 

bottleneck. The introduction of a "keep alive" header allows control of user accesses. Moreover, the IP address spaces are 

sessions to be kept open and used for multiple HTTP 50 partitioned into five different classes. Care should therefore 

request/response activities. be taken in designing good hashing function. 

The web servers have also undergone improvements: HTTP is a stateless protocol. A web server obtains every- 

first-generation servers handled 20 transactions/sec. based thing it needs to know about a request from the HTTP 

on one process per transaction. The major overhead was due request itself. After the request is serviced, the server can 

to a large process fork time for a new transaction. This was 55 forget the transaction. Thus, each request in HTTP is dis- 

avoided by pre-forking multiple processes and using a jointed. If all the servers are identical (or see the same file 

dispatcher to distribute transactions among them to achieve system using a distributed file system), the server from 

server performance of 100 transactions/sec. The "keep which the request is served is of little relevance to the client, 

alive" HTTP feature, along with the multi-threaded archi- The choice of a physical server itself is immaterial to the 

lecture of only one process, allows the server to handle more 60 transactions. An HTTP transaction is an aggregation of one 

than 250 transactions/sec. However, the current trend indi- or more TCP sessions. Based on this principle, different TCP 

cates that the popular sites will incur a significantly higher sessions can be allocated to different servers without the 

number of server transactions per second in the near future. knowledge of whether or not all the TCP sessions belong to 

This requires more powerful web servers which may be the same HTTP transaction. The present invention realizes 

developed by improving different components of a web 65 this TCP-based switching by the use of an intermediary 

server (e.g., CPU speed, disk performance, file system entity called a depot to perform these functions of session 

performance, performance of TCP/IP, server software archi- allocation. Thus, the TCP-based server switching allows a 
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nice granularity for load balancing among multiple servers. servers. The depot comprises a packet analysis block for 
It is also envisaged that this concept of forwarding different analysing and forwarding packets of one or more informa- 
sessions to different servers can be applied to similar multi- tion transfer sessions between a client and one of the servers 
server architectures of telecommunications networks. and a session management block for switching one or more 

5 information transfer sessions among a plurality of servers 
OBJECTS OF THE INVENTION according to a preset criteria including the states of each 

It is an object of the invention to provide a server system inform f on t™sfcr session The depot further includes a 
which has a high performance. protoco1 stale trackin S biock for monitoring the states of the 

° r sessions. 

It is another object of the invention to provide a commu- 10 

nications architecture which allows the construction of a BRIEF DESCRIPTION OF THE DRAWINGS 

scale able and high performance server system. __ _ . , , , , 

n • c _*u i ■ . r% • ... . j , , FIG. 1 shows multiple server systems of known architec- 

It is a turther object of the invention to provide a method 

of and an apparatus for efficiently managing the resources of - , , 

a multiple server system. 15 FIG ' 2 shows anolher mulll P le server s Y ste m of known 

architecture' 

It is still another object of the invention to provide a „ 10 _ / t . „ 
method of and an apparatus for efficiently managing the F1 ^. 3 shows schematically a client-server server system 
resources of a multiple server system by the use of an according to one embodiment of the invention; 
intermediate entity. FIG. 4 is a functional block diagram of an intermediary 

It is yet a further object of the invention to provide a 20 device accordin S to OQe embodiment of the invention; 
method of and an apparatus for efficiently managing the FIG * 5 is a schematic illustration of the invention as 
resources of a multiple server system by the use of more than applied in the ATM environment according to one embodi- 
one instance of the intermediate entity for improved robust- ment; 

ness. FIG. 6 shows protocol stacks of the architecture shown in 

• 25 FIG. 5; 

SUMMARY OF THE INVENTION FIG 7 ^ a fuflCtional bJock diagram of lhe embodimem 

Briefly stated, the invention resides in a client-server shown in FIG. 5; 
environment of a telecommunications network, where infor- FIG. 8 is a schematic illustration of a yet another embodi- 

mation resources are replicated among a plurality of servers 30 naent of the invention; and 

and a plurality of clients have access to any one or more of FIG. 9 shows schematically a yet further embodiment of 
the servers for desired information resources. According to the invention as applied in the proxy server environment, 
one aspect, the invention is directed to a method of effi- 
ciently utilizing the plurality of servers. The method com- DETAILED DESCRIPTION OF PREFERRED 

prises steps of performing a transaction between one of the 35 EMBODIMENTS OF THE INVENTION 

plurality of clients and the plurality of servers by way of an FIG> 3 shows schematically the invention according to 

intermediary function called a depot, each transaction com- one embodiment. Aclient 30 is communicating with a server 
prising one or more information transfer sessions, switching 32 through a telecommunications network which can be any 
at the depot the plurality of sessions among the plurality of transport network such as telephone networks, data 

servers so that during each session transfer of the informa- 40 networks, ATM networks, LANs, or a combination thereof, 
tion resources is performed between one client server pair. WWW or Internet runs on top of the transport network. 

According to another aspect, the invention resides in a Typically, a client sends the server 32 a request for infor- 
client-server environment of a telecommunications network, mation stored therein and the server replies with the desired 
where information resources are replicated among a plural- information. The desired information, however, may require 

ity of servers and a plurality of clients have access to anyone 45 more than one session (e.g., TCP session). In the multiple 
or more of the servers for desired information resources server environment, servers 34 and 36 contain identically 
through transactions using at least two layers of protocols, stored information as that stored in server 32. According to 
one stateless protocol layer upon another stateful protocol the one embodiment of invention, an intermediate entity 38 
layer. The invention is therefore directed to a method of sits transparently between the client and the servers. The 

efficiently utilizing the plurality of servers during the trans- 50 intermediate entity 38 is called a "depot" throughout this 
actions. The method comprises a step of performing each specification and is the heart of the invention. When the 
transaction between one of the plurality of clients and the depot receives a request for information, it distributes the 
plurality of servers by way of a depot, each transaction TCP sessions among multiple servers based on the server 
comprising one or more information transfer sessions. The load balancing criteria. In actual implementation, the depot 

invention further includes the following steps performed at 55 can be realized by the use of appropriate software, hardware 
the depot: inspecting all packets of each information transfer or a combination of the both. It should also be noted that the 
session, forwarding packets of existing information transfer concept of session distribution among the servers can be 
sessions to a correct server, detecting packets of a new implemented in other multiserver environments which use 
information transfer session, selecting a server among the protocol stacks of similar configurations, 

plurality of servers, and forwarding all the packets of the 60 The depot is a forwarding mechanism and forwards any 
new information transfer session to the selected server so particular packet to the assigned server using a map. All 
that during each of the existing and new information transfer clients access the server system using the depot's IP address, 
session transfer of the information resources is performed The distribution of TCP sessions among multiple servers by 
between each of a plurality of client-server pairs. me depot remains transparent to the client. A TCP session 

In accordance with another aspect, the invention is 65 consists of multiple TCP packets. All TCP packets of a given 
directed to a depot for transferring one or more information TCP session are served by the same server. Since the 
transfer sessions between a client and among a plurality of forwarding of all TCP packets of a given TCP session must 
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go to the same server, a map between Ihe IP address and port otherwise, the introduction of an intermediary such as the 

number of the client and the identity of the server is depot may disrupt the normal TCP logic between the two 

maintained. The entry of this map is to be maintained in the end systems. The crucial function of the state tracking 

depot as long as required by the TCP protocol This entry is routine in the depot involves the creation and deletion of the 

preserved in the depot as long as there is a possibility of the 5 table entries. Several types of TCP/IP packets can initiate the 

arrival of a TCP packet from a client or a server. The depot creation and deletion of table entries. In the TCP state 

does not generate any TCP packets nor does it discard any transition diagram, there are eleven states. For the client- 

unless they are anomalous enough that there is no way to server model, however, the number of possible states is 

assign them to a server. reduced. This reduces the state tracking requirement in the 

A depot has the following functions: 10 depot. However, out-of -sequence situations can occur in the 

1. inspect all packets in both directions at IP and TCP levels; Internet, and different permutations of state transitions can 

2. choose a server based on load balancing criteria for a new arise which slightly increase the complexity of the state 
TCP session; tracking routine. Moreover, in wide-area networks, packet 

3. forward TCP packets for existing sessions to the already loss is possible. Therefore, any packet received by the depot 
chosen server; 15 does not guarantee the reception of the packet by the other 

4. forward TCP packets from servers to clients; end. As a result, according to the invention, packets in both 

5. clean up the mapping entry when TCP sessions end; and directions are analysed in order to get a complete view of the 

6. watch for and handle anomalous TCP packets. state of the session and the state tracking function of the 
All these functions are performed in a depot and FIG. 4 depot simply guesses the current TCP states of the client and 

shows its functional block diagram. Referring to FIG. 4, the 20 server by investigating the "flag" information of all arriving 

depot includes TCP packet forwarding block 40 for TCP TCP packets. The states guessed at the depot may be 

packet analysis and forwarding. This block inspects all different from the actual states at the client and server, if a 

packets in both directions at IP and TCP levels, forwards TCP packet is dropped or corrupted in the network between 

TCP packets of existing sessions to the already chosen the depot and the client or server. 

server, and forwards TCP packets from servers to the clients. 25 The following is a detailed description of the handling of 

For a new session, this block identifies the TCP session setup various TCP packets according to one embodiment of the 

request and forwards information to session management invention, 

block 42. The session management block 42 chooses a SYN 

server among a pool of available servers. This block is A SYN packet header (client IP address and port) is 

preloaded with the next server to be allocated. This pre-load 30 matched with the entries in the primary table and then with 

is done by running a load balancing algorithm such as shown the secondary table. If no match is found (i.e., arrival of a 

at 44 in the figure on the basis of knowledge of the server new session), a new entry is created in the primary table and 

statistics and perhaps network states, or simply by round- based on the load balancing criterion, a server is allocated 

robin. Server probing 48 can be performed periodically to for the session. If a match is found (i.e., duplicate SYN), the 

obtain the server statistics. The TCP packets carrying data 35 packet is forwarded to an already allocated server, 

are forwarded by the TCP packet forwarding block 40 by FIN, PSH, URG 

identifying the entry of a map between the server and the The match is found from the primary or secondary table 

client. At TCP state tracking block 46, TCP states of both the and forwarded to the appropriate server. If no match is 

clients and the servers are being monitored to facilitate the found, the packet is dropped, 

management of the TCP session, e.g., opening/closing and 40 ACK 

control functions. All ACK packets are forwarded to the server if an entry 

The depot maintains the map 50 of all the active TCP is found in the table. If the ACK packet causes the state 

sessions in a table called the primary table. Each session transition to the HME_WAIT state, the entry is moved from 

between a client and a server is identified by the combination the primary table to the secondary table for a 2 MSL 

of the client's IP address and port number. 45 time-out. 

A typical primary table entry has the following: client IP RST 

address and port number, server identity, TCP states and All reset packets are validated by checking if the sequence 

related parameters (ack number, seq number etc.). The depot number is in the window. If the state is SYN-SENT, the RST 

maintains another list of sessions in a table called the is considered valid if the ACK field acknowledges the SYN. 

secondary table. The entries of the primary table are deleted 50 If the RST is valid, the entry from the primary table is moved 

at the close of sessions under normal or anomalous condi- to the secondary table. 

tions. The secondary table entries are maintained for a 2 The exception conditions are handled in the following 

MSL (maximum segment lifetime) period. The duration of manner. 

a 2 MSL differs for different implementations but, in this If an entry in the primary table is inactive for a long time 

embodiment, a period of 2 minutes is chosen as an example. 55 (e.g., more than 20 mins.), the entry is moved to the 

The 2 MSL is important since a TCP packet may arrive after secondary table. This is necessary since a client or server 

a session is closed, due to variable delay at the network. may crash without proper termination of a TCP session, 

The reason for maintaining two separate tables at the causing an entry to the primary table to remain forever, 

depot is to reduce the search space for finding a match on If an entry in the tables (primary and secondary) is not 

arrival of a TCP packet. The number of entries in the 60 found on arrival of a packet other than SYN, the packet is 

secondary table is very large since the entries are maintained dropped. This is necessary since the depot does not know the 

for a 2 MSL period. The entries in the primary table are for destination for the packet. 

active TCP sessions only. Most of the incoming TCP packets The depot does not cause any interruption of the ongoing 

should find a match in the primary table. TCP sessions between the clients and the servers. This is 

A TCP session is statefiil within each session between a 65 because the depot forwards every packet, irrespective of the 

client and a server. Therefore, the depot should handle all states guessed at the depot, as long as a table entry is found, 

arriving TCP/IP packets as transparently as possible; For example, a RST may be lost in the network between the 
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depot and client but the entry is still maintained in the point to each server, which provides traffic management and 

secondary table, which enables the forwarding of any any necessary IP/ATM conversion. FIG. 8 shows all the 

retransmitted RST. In one embodiment, the forwarding from servers and Internet connections accessing the ATM network 

the depot to the servers is achieved by directing packets to via edge nodes. The edge nodes communicate with each 

physical ports of the device containing the depot function 5 ot hcr over independent virtual circuits. Therefore, depot 82 

where each physical port connects to one server. fa alsQ an edge node managing the Intcrnet connection that 

According to a further embodiment the depot system of has the additional function of routing incoming packets to 

the invention is implemented in the ATM environment. In me correc t server 

this embodiment, an ATM network is used as a transport According to a' yet further embodiment of the invention, 

mechanism to provide distributed WW services. The 10 me depot system of management is applied to the 

network model is shown in FIG. 5. On the ATM network, server architeclure . p r0X y servers are used to reduce 

multiple, information homogeneous, but potentially geo- me network load and lat b migrating the work load 

graphically distributed web servers 50 are provided. An close tQ me c jj ents 

Internet web client 52 can access information via the routes A proxy server is an app ii cation gateway which operates 

shown on the diagram. Adepot 54 of the invention provides 15 at the HyperText Transfer Protocol (HTTP) layer. The basic 

the interface between an Internet client and an ATM network fo nciion of a proxy server ^ almost idcntical {Q a Hrrp 

server. An incoming packet from the client will be seg- scrver in traa sferring client requested documents, 

mented and encapsulated into AAL5 cells at the depot Furthermore, it is able to interpret and modify a HTTP 

without the requirement of modifying either the IP or TCP request before forwarding it to the end WWW server. This 

headers. The cells (packets) are earned sequentially in a 20 is because the proxy server has the caching function. If there 

virtual channel. ^ a cacne hit, it delivers the found documents to the client 

Uke the earlier embodiment, the depot ideally sits trans- local] thus reducing the network load and transmission 

parently between the clients accessing from the Internet and latency 

the servers on the ATM networks. The depot employs a FIG. 9 shows schematically the concept of another 

TCP-based switching mechanism. It examines all TCP/IP 25 embodiment of the invention as applied to the proxy server 

packets and sends them to the ATM network from the manag ement system. In FIG. 9, a depot proxy system is 

Internet and vice versa. FIG 6 shows the protocol stack located between an intranet and ^ , nternet The d t 

structure of the system model On the depot, there are TCP distributes sessions among a pool of proxy servers based on 

and IP stacks but the TCP session is only observed and not load balancing or other criteria< ^ function of the depot 

terminated. 30 0 f me proxy system is identical to those described earlier. 

Therefore, the depot of the embodiment performs the Therefore, for a new session, the packet analyzer identifies 

same functions as those described earlier. In addition, the TCP session setup request and forwards the information 

however, it must perform the following functions: to the session managem em block. If the session is already 

transform IP packets to ATM cells and vice versa; allocated and recognized by the packet analyzer, it will read 

map the QoS parameters from IP to ATM where appli- 35 in the correct proxy identity from its tables and forward the 

cable; and packet onward. Packets in the reverse direction are also 

participate as necessary in the ATM network manage- analyzed in order to have a complete view of the state of the 

ment. session. 

Similar to FIG. 4, FIG. 7 shows a functional block This proxy architecture can achieve the following similar 

diagram of the depot of this embodiment. The depot iden- 40 goals: 

tifies the flow (client IP and port) from the TCP/IP packets 1. scaleable proxy server arrangements; 

and maps the corresponding VPI/VCI in its table entries. At 2. high availability of information service; and 

the IP/ATM adaptation block, the depot performs segmen- 3. dynamic load balancing of traffic loads to different proxy 

tation and encapsulation of packets into AAL5 cells without servers. 

the requirement of modifying either the IP or TCP headers. 45 To achieve scaleability, more proxy servers can be 

Likewise, it also perform the reverse functions of ATM cells attached to the depot. This is because the depot bandies only 

from the ATM network. For a new session, the packet simple functions. When a depot operates to its limitations, it 

analyzer identifies the TCP session setup request and for- is also possible to add a new depot with a new IP address 

wards the information to the session management block. The under the same alias name. With the newly added depot, 

next server to be allocated is selected, as described earlier. 50 another cluster of proxy servers can be created on the 

Packets are kept contiguous and are all sent over the same network. 
VC. 

If the session is already allocated and recognized by the EXPERIMENTAL RESULTS 

packet analyzer, it will read the correct VPI/VCI from its A Pentium PC running a NetBSD operating system is 

tables and forward the packet onward. Packets in the reverse 55 used for the software implementation of the depot. The 

direction are also analyzed in order to have a complete view client's HTTP requests are generated using benchmark soft- 

of the state of the session. ware from Zeus Corporation. The clients and servers are 

In another embodiment, the forwarding is achieved by connected to the depot using 10 Mbps Ethernet. The same 

encapsulating the IP frames in a frame transport, such as test file is retrieved 1000 times for each test. The number of 

Ethernet or Frame relay, using the addressing of that frame 60 simultaneous HTTP requests are varied to study the depot 

transport to identify individual servers. behaviour. Traffic is forwarded by the depot to two identical 

In another embodiment, the forwarding is achieved by NCSA/1.5.1 Web servers in a round robin fashion. Server 

modifying the header information of the TCP/IP packet and throughput (total bytes tranferred per second) is measured 

redirecting it over an IP network. for each test case. The total number of bytes of data and the 

In FIG. 8, another embodiment is shown. According to 65 HTTP headers divided by the time taken to transfer indicates 

this embodiment, servers may or may not be ATM-based me server throughput. The number of HTTP requests served 

hosts and therefore an edge node 80 is required as an access by the server per second is also measured. Both measures 
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include a variable network delay. To minimise the variable 
network delay, the experiment is performed at a time when 
the network is very lightly loaded. The variation of the 
server throughput and HTTP requests served per second 
with an increased rate of client requests are good indicators 5 
of server performance. 

TABLE 1 





file size: 


file size: 


file size: 


Con- 


100 bvtes 


1 Kbvtes 


10 Kbvtes 


current 


single 




single 




single 




sessions 


server 


invention 


server 


invention 


server 


invention 


1 


18.22 


17.46 


66.58 


56.67 


52.13 


51.98 


5 


29.21 


44.62 


99.26 


80.75 


151.85 


168.18 


10 


21.1 


55.13 


94.83 


99.81 


226.80 


24453 


15 


10.18 


39.5 


65.75 


101.73 


223.1 


277.06 


20 


13.17 


29.86 


42.64 


111.93 


251.24 


289.91 


25 


6.46 


30.00 


33.86 


117.53 


233.73 


297.89 


100 


6.55 


12.23 


X 


37.59 


X 


282.81 


TABLE 2 




file size: 


file size: 


file size: 


Con- 


100 bytes 


1 Kbvtes 


10 Kbvtes 


current 


single 




single 




single 




sessions 


server 


invention 


server 


invention 


server 


invention 


1 


64.61 


61.92 


55.16 


46.95 


5.00 


4.99 


5 


103.48 


158.23 


82.19 


66.88 


14.55 


16.13 


10 


74.59 


194.70 


78.31 


82.55 


21.68 


23.44 


15 


35.73 


138.95 


54.03 


84.21 


21.35 


26.53 


20 


39.03 


104.62 


34.87 


92.29 


23.97 


27.67 


25 


22.48 


105.22 


27.55 


97.23 


22.40 


28.49 


100 


22.41 


40.79 


X 


28.51 


X 


25.69 



10 



15 



20 



25 



30 



35 



40 



45 



Table 1 shows the server throughput in Kbytes/sec. The 
experiment is performed with three different file sizes: 100 
bytes, 1 Kbyte and 10 Kbytes. The number of simultaneous 
client HTTP requests are varied from 1 to 100. The total 
number of client HTTP requests for each test is 1000. The 
experiment is performed with a single server system and a 
depot system with two servers. Table 2 shows the number of 
served HTTP requests per second under the same experi- 
mental setup. 

For one client HTTP request (any file size) at a time, the 
depot system is always slower than the single server. This is 
due to the store and forward delay introduced at the depot. 
For multiple simultaneous HTTP requests, however, the 
HTTP requests are served in parallel by two servers. In 5Q 
general, the depot system shows improved throughput and 
served HTTP requests per second. Another reason for per- 
formance improvement is the reduced workload at each 
server due to the distribution of HTTP requests. For 100 
bytes and 1 Kbyte files, the single server performance 5S 
degrades quickly with the increased number of simultaneous 
sessions. Performance with the depot system is consistently 
better than the single server system. For the 10 Kbytes file, 
the single server performance remains flat. However, the 
performance with the depot system is better than the single $Q 
server system. 

For 100 simultaneous HTTP requests, the single server 
system could complete the test for only a 100 bytes file size. 
The depot system could complete the tests for all three file 
sizes. 

It is found that the throughput and HTTP requests served 
per second with the depot system are more than double those 
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of the single server system with a large number of simulta- 
neous sessions. For example, the number of HTTP requests 
served per second for the test case of 1 Kbyte file size with 
20 simultaneous sessions is 92.29. This is an improvement 
of 2.65 over the single server system. This linear improve- 
ment is due to the halved number of simultaneous sessions 
(i.e., 10) on each server. The number of HTTP requests 
served per second by the single server system with 10 
simultaneous HTTP requests is 78.31. Thus, maximum 
served requests of (2*78.31) 156.62 is theoretically attain- 
able. 

Thus it is shown that the server throughput and serviced 
requests per second can be improved using multiple servers 
according to the invention. 

In the multiple server system, if one server goes down, the 
depot transfers all subsequent sessions to the other servers in 
the same cluster. There are several methods to detect server 
failure. A depot may assume the failure of a server if it does 
not receive an ACK from the server for several seconds after 
it sends a message to the server. After that, the depot starts 
polling the failed server on a regular basis to check if the 
server program has restarted. Another method that the depot 
uses is the passive reception of server statistics as an 
indication of the operability of a server program. If the depot 
does not receive any information for a period of time, then 
the depot will put this server into a dead-list and new 
incoming TCP requests will not be routed to this server. 
When server statistics are received from this server again, 
the depot will then put it back into an alive-list. However, 
there is difficulty in identifying the current state within each 
TCP session such as the number of bytes sent, the current 
window size and sequence number etc. Thus, the lost TCP 
sessions from a faulty server are simply scrapped. 

The depot reliability is also very important since all the 
traffic is concentrated at the depot. Standby sparing is a 
simple method of providing fault tolerance in the system. 
There can be hot or warm standby. A sparing standby depot 
machine is always in alert mode to assume normal depot 
operations upon detecting a failure in the operating depot. 
For hot standby, the replica actively monitors all input 
streams from both the client and server. Both operating 
machine and replica have virtually identical TCP state 
information about all sessions. Detection of depot failure 
initiates changeover of control. 

For warm standby, the operating machine periodically 
sends the state information to the replica to update the 
standby's database. In this situation, the two databases shall 
not be the same most of the time, but the goal is to reduce 
the number of affected TCP sessions when failure occurs. 
Then the standby replica will automatically be activated on 
fault detection. 

What is claimed is: 

1. In a client-server environment of a telecommunications 
network, where information resources, which are accessed 
by HTTP protocol, are replicated among a plurality of 
servers and a client has access to any one or more of the 
servers for desired information resources, a method of 
efficiently utilizing the plurality of servers comprising steps 
of: 

performing an HTTP transaction by HTTP protocol 
between the client and the plurality of servers, the 
HTTP transaction comprising one or more TCP con- 
nection sessions which are conducted by way of a 
depot; 

storing, at the depot, information about the TCP connec- 
tion sessions and parameters in a primary and a sec- 
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ondary map, the information in the primary map being 
deleted at the close of sessions and the information at 
the secondary table being maintained for two maximum 
segment lifetimes (MSL); 

transparently forwarding TCP connection sessions of the 5 
HTTP transaction at the depot in both directions to and 
from the client and the plurality of servers; 

monitoring parameters indicative of either or both of load 
and operability of the servers; 

detecting transition states of the TCP connection sessions 10 
of the HTTP transaction at the depot; and 

switching at the depot the plurality of TCP connection 
sessions of the HTTP transaction among the plurality of 
servers based on the detected transition states and the 15 
monitored parameters of the servers so that during the 
HTTP transaction, transfer of the information resources 
is performed by HTTP protocol between the client and 
one or more servers among the plurality of servers. 

2. The method according to claim 1 wherein there are a 2Q 
plurality of depots, comprising further steps of: 

monitoring parameters indicative of operability of all the 
depots; and 

upon detection of inoperability of one depot, continuing 
all the functions of the failed depot at any of the 25 
remaining depots. 

3. The method according to claim 1 wherein the depot is 
connected to one or more servers through an ATM network, 
comprising a further step of: 

performing the TCP connection sessions in ATM cells 30 
through the ATM network. 

4. The method according to claim 3 wherein there are a 
plurality of depots, comprising further steps of: 

monitoring parameters indicative of operability of all the 
depots; and 35 

upon detection of inoperability of one depot, continuing 
all the functions of the failed depot at any of the 
remaining depots. 

5. The method according to claim 1 wherein the depot is 
connected to one or more servers through a local area 40 
network or a frame relay network, comprising further steps 
of: 

performing the TCP connection sessions in their appro- 
priate format through the local area network or the 
frame relay network. 45 

6. The method according to claim 5 wherein there are a 
plurality of depots, comprising further steps of: 

monitoring parameters indicative of operability of all the 
depots; and 

upon detection of inoperability of one depot, continuing 
all the functions of the failed depot at any of the 
remaining depots. 

7. The method according to claim 1, wherein the depot is 
connected to one or more servers by way of a plurality of 55 
proxy servers, comprising further steps of: 

switching at the depot the TCP connection sessions 
among the proxy servers which in turn send the HTTP 
transaction to the servers; and 

cacheing information resources contained in the the 60 
HTTP transaction at the proxy servers. 

8. The method according to claim 7, comprising further 
steps of: 

monitoring the status of all the proxy servers; and 

for each TCP connection session, selecting one proxy 65 

server among all the proxy servers based on their 

monitored status. 
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9. The method according to claim 8 wherein there are a 
plurality of depots, comprising further steps of: 

monitoring parameters indicative of operability of all the 
depots; and 

upon detection of inoperability of one depot, continuing 
all the functions of the failed depot at any of the 
remaining depots. 

10. In a client-server environment of a telecommunica- 
tions network, where information resources, which are 
accessed by HTTP protocol, are replicated among a plurality 
of servers and a plurality of clients have access to any one 
or more of the servers for desired information resources 
through a plurality of HTTP transactions, each HTTP trans- 
action using multiple TCP connection sessions, a method of 
efficiently utilizing the plurality of servers during the HTTP 
transactions, comprising steps of: 

performing each HTTP transaction on HTTP protocol 
between any one of the plurality of clients and the 
plurality of servers, each HTTP transaction comprising 
one or more TCP connection sessions which are con- 
ducted on TCP/IP through a depot; and 
at the depot, performing the following steps: 
storing information about the TCP connection sessions 
and parameters in a primary and a secondary map, 
the information in the primary map being deleted at 
the close of sessions and the information at the 
secondary table being maintained for two maximum 
segment lifetimes (MSL); 
inspecting all TCP/IP packets to detect a state transition 

of the TCP connection sessions; 
transparently forwarding TCP/IP packets of existing 

TCP connection sessions to a correct server; 
detecting TCP/IP packets of a new TCP connection 
session; 

monitoring parameters indicative of either one or both 
of load and operability of the servers; 

selecting a server among the plurality of servers based 
on their monitored parameters and the state transi- 
tion; and 

transparently forwarding all the TCP/IP packets of the 
new TCP connection session to the selected server so 
that during each of the existing and new TCP con- 
nection sessions, transfer of the information 
resources of the HTTP transactions is performed 
between any one of the clients and one or more 
servers by one or more new and existing TCP 
connection sessions. 

11. The method according to claim 10 wherein there are 
a plurality of depots, comprising further steps of: 

monitoring parameters indicative of the operability of all 

the depots; and 
upon detection of inoperability of one depot, continuing 

all the functions of the failed depot at any of the 

remaining depots. 

12. The method according to claim 10 wherein the depot 
is connected to one or more servers through an ATM 
network, comprising a further step of: 

performing the TCP connection sessions in ATM cells 
through the ATM network. 

13. The method according to claim 12 wherein there are 
a plurality of depots, comprising further steps of: 

monitoring parameters indicative of the operability of all 

the depots; and 
upon detection of inoperability of one depot, continuing 

all the functions of the failed depot at any of the 

remaining depots. 
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14. The method according to claim 10 wherein the depot 
is connected to one or more servers through a local area 
network or a frame relay network, comprising further steps 
of: 

performing the TCP connection sessions in their appro- 5 
priate format through the local area network or the 
frame relay network. 

15. The method according to claim 10, wherein the depot 
is connected to one or more servers by way of a plurality of 
proxy servers, comprising further steps of: 10 

switching, at the depot, the TCP connection sessions 
among the proxy servers which in turn send TCP 
connection sessions to the server; and 

cacheing information resources contained in the TCP 15 
connection sessions at the proxy servers. 

16. The method according to claim 15, comprising further 
steps of: 

monitoring the status of all the proxy servers; and 

for each TCP connection session, selecting one proxy 20 

server among all the proxy servers based on their 

monitored status. 

17. The method according to claim 16 wherein there are 
plurality of depots, comprising further steps of: 

monitoring parameters indicative of the operability of all 25 

the depots; and 
upon detection of inoperability of one depot, continuing 

all the functions of the failed depot at any of the 

remaining depots. 

18. In a client-server environment of a telecommunica- 30 
tions network, where information resources are replicated 
among a plurality of servers and a plurality of clients have 
access to any one or more of the servers for desired 
information resources through an HTTP transaction using at 
least two layers of protocols, one stateless protocol layer 35 
upon another stateful protocol layer, the HTTP transaction 
running on the stateless protocol layer of HTTP protocol and 
being made up of one or more TCP connection sessions 
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which run on TCP/IP layer, a depot for transferring one or 
more TCP connection sessions between a client and among 
a plurality of servers, comprising: 

a primary and a secondary map for storing information 
about the TCP connection sessions and parameters, the 
information at the primary map being deleted at the 
close of sessions and the information at the secondary 
map being maintained for two maximum segment 
lifetimes (MSL); 

a packet analysis block for analysing and transparently 
forwarding TCP/IP packets of one or more TCP con- 
nection sessions of the HTTP transaction between the 
client and one of the servers; 
a session management block for switching one or more 
TCP connection sessions of the HTTP transaction 
among the plurality of servers according to a preset 
criteria including the TCP/IP states of each TCP con- 
nection session; and 
a protocol state tracking block for monitoring the TCP/IP 
states of the TCP connection sessions of the HTTP 
transaction. 

19. The depot according to claim 18, further comprising: 
a map which contains data concerning either one or both 

of load and operability of the plurality of servers; and 
the session management block for switching one or more 
TCP connection sessions of the HTTP transaction 
among the servers based on the data storedd in the map. 

20. The method according to claim 14 wherein there are 
a plurality of depots, comprising further steps of: 

monitoring parameters indicative of the operability of all 

the depots; and 
upon detection of inoperability of one depot, continuing 
all the functions of the failed depot at any of the 
remaining depots. 

* + * * * 
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